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PREFAOE 


This  report  is  the  eleventh  in  a series  produced  by  the  Elec- 
tronics Personnel  Research  group.  Technical  Reports  Ho.  9 through 
12  are  especially  oriented  to  the  domain  of  electronics  trouble 
shooting,  and  their  titles  are  listed  belovt 

9.  A Methodological  Study  of  Electronics  Trouble  Shooting 
” Bklilt  I.  AatloneTe  ^or  and  description  0: i tEc 

Halt  lpl e^U.t ernat  ive  Symtol io  Trouble  ^Sooting  fast. 

A description  of  a now  type  of  teat  format  designed 
for  measuring  some  aspects  of  trouble  shooting  skill, 
and  a discussion  of  the  conception  of  trouble  shoot- 
ing on  which  it  is  based. 

10.  A Methodological  Study  of  Electronics  Trouble  Shooting 

gkiiir  Hr  Inierconparlsone  0/  tne  MAStS  feats, 
a Joi-S  ample  9est,  andien  deference  Tests  JLdainis- 

teF^TTfleerWs. 

A report  of  the  results  of  the  administration  of  two 
forms  of  the  HASTS  Tests,  its  progenitor  job-sample 
test,  and  a battery  of  achievement  and  ability  ori- 
ented reference  testa  to  a sample  of  BTs  from  ships 
undergoing  repairs  in  the  Long  Beach  Haval  Shipyard. 

11.  The  AUTCKASTSl  An  Automatically-Recording  Test  of  Elec- 

tron lc»"TrouSle  Shooting.  — — 

A detailed  description  of  an  automatically-recording 
version  of  the  HASTS  Test.  Mechanical  and  administra- 
tive features  of  the  test  are  presented.  The  test 
problems  and  proposed  scoring  procedures  are  discussed, 
and  a variety  of  applications  are  suggested. 

12.  An  Experimental  Battery  for  Measurement  of  the  Profic iency 

of  dlec tronics  Technicians. 

A report  of  the  results  of  administering  the  AUTOMASTS 
Test  and  a battery  of  nine  printed  electronics  tests 
to  a large  sample  of  electronics  maintenance  personnel 
from  the  Pacific  Pleat. 
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ABSTRACT 


This  report  is  one  of  a series  concerned  with  the  analysis  and 
measurement  of  electronics  trouble  shooting  behavior.  An  automatical- 
ly recording  testing  device  called  the  AUT015ASTS  is  described  in  de- 
tail. Problems  used  in  the  device  and  procedures  for  administering 
the  test  are  presented.  Proposals  for  scoring  the  response  records 
are  introduced  along  with  suggestions  for  research  applications  of 
the  instrument. 
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THE  AUT011AST8»  AH  AUTOHATICALLY-BJtCQRDING  TEST 


OT  ELECTRONICS  TROUBLE  SHOOTIHG 
I.  IHTRODITCTIOH 

The  development  of  modern  electronic  systems  has  necessitated 
the  development  of  procedures  for  evaluating  and  analyzing  the  per- 
formance of  men  engaged  in  repair  of  electronic  equipment.  This 
report  describes  an  experimental,  automatically- recording,  profi- 
ciency test  used  to  assess  the  capabilities  of  naval  electronics 
repairmen.  Since  the  testing  devioe  is  new,  it  is  described  in  de- 
tail. The  electronics  "troubles,*  or  problems,  used  as  subject  mat- 
ter during  the  initial  tryout  of  the  testing  machine  are  described, 
and  the  procedure  for  administering  the  test  is  given.  Mention 
also  1b  made  of  the  possibilities  for  scoring  the  performance  re- 
cords, The  results  of  the  administration  of  this  test  along  with 
a battery  of  more  conventional  tests  will  be  presented  in  Technical 
Report  Ho.  12. 

Historical  Background 

The  AUTOiiASTS  Test  has  evolved  through  several  stages.  Its 

immediate  predessor  is  the  liaSTS  Test.  That  test  and  its  devel- 

1 

opment  are  described  in  Technical  Report  No.  9»  The  AUTCIiASTS 

"'"A  Iiethodo  logical  Gtudy  of  Electronic  a Trouble  Shooting  Skill} 
I,  RaTionalo  for  and  Rescript  ion  of  tke"  l^uitiple^Xlternativ? 
Symbolic  Trouble  Shooting  Test.. 


Test  is  essentially  a refined,  and  automatically-recording  form  of  the 
MASTS  Test  and  is,  therefore)  based  upon  the  earns  conception  of  trouble 
shooting  and  test  rationale* 

In  the  summer  of  1953  the  MASTS  Test  was  individually  administered 

to  a sample  of  36  electronics  technicians  from  the  Pacific  Fleet  at  the 

Long  Beach  Koval  Shipyard,  Sach  man  took  the  test  along  with  trouble 

shooting  problems  in  actual  electronic  equipment.  The  results  of  that 

2 

study  are  reported  elsewhere.  For  the  purpose  of  the  present  report 
it  is  sufficient  to  say  that  the  results  indicated  that  trouble  shoot- 
ing scores  on  the  MASTS  Test  were  rather  good  predictors^  of  trouble 
shooting  scores  on  actual  electronic  equipment.  At  the  conclusion  of 
the  field  study  it  was  decided  to  construct  an  automatically-recording 
form  of  the  test  and  to  adapt  to  the  automatio  model  some  of  the  same 
problems  that  had  been  tried  out  during  the  Long  Beach  testing. 


The  Advantages  of  an  Automat  ioally-Becordlng  Test 

There  are  a number  of  advantages  to  on  automatically-recording 
test  as  compared  with  the  same  test  administered  on  the  prototype 
equipment,  i, e. , the  actual  electronic  gear.  Most  of  these  are  obvi- 
ous, but  a few  are  worthy  of  mention.  One  of  the  main  advantages  is 


a 

"See  our  Technical  Beport  Ho.  10  entitled,  HA  Methodological 
Study  of  Blectronica  Trouble  Shooting  Skill!  II*"’  tint ercompar is ons 
of  the"TI&J>!i&  !test , a ^ob-^anple  TeatT  and  fen  Reference  Tests  Admln- 

Hl SSTESSTSfi*11 — 

■x 

Bank  order  correlations  between  the  job  sample  total  score  and 
the  ilASTS  total  score  range  from  0,60  (where  the  score  is  the  number 
of  problems  solved)  to  0.77  (where  the  score  is  the  median  time  of 
solution) . 


that  the  test  my  be  simultaneously  administered  to  as  many  experi- 
mental subjects  as  tho  number  of  testing  machines  available.  The 
implications  of  this  may  not  be  striking  to  one  who  has  not  at- 
tempted to  obtain  complete  response  records  of  the  performance  of 
a man  while  he  is  repairing  a piece  of  equipment , but  those  who  have 
done  so  aan  confirm  the  arduous  nature  of  the  task* 

Under  such  circumstances  each  observer  has  to  be  carefully 
trained  in  the  techniques  of  observing  and  recording  as  well  as  in 
the  teohnlcal  subject  matter  of  the  job.  Only  one  experimental  sub- 
ject's performance  may  be  observed  and  recorded  at  a time.  In  order 
to  test  more  than  one  man  at  a time  on  the  actual  equipment,  each 
experimental  subject  has  to  be  provided  with  a set  of  electronic 
gear,  reference  materials,  and  test  equipment.  Hash  subject  re- 
quires the  services  of  a trained  observer.  Because  of  these  and 
other  practical  requirements,  the  number  of  cases  studied  in  this 
manner  is  usually  so  small  that  adequate  statistical  treatment  of 
the  response  records  is  difficult. 

When  automatically- recording  testing  machines  are  employed, 
one  observer  can  supervise  several  machines  at  one  time  and  the 
collection  of  data  for  a large  number  of  cases  becomes  ec onomically 
feasible.  It  is  unnecessary  for  the  administrator  to  participate  in 
the  testing  in  any  way  except  to  change  the  problem  in  the  machines 
and  generally  supervise  the  men  talcing  the  test,  A3  a result,  rel- 
atively untrained  and  electronic?’ lly  naive  test  administrators  can 
be  used. 

Another  advantage  is  that  with  an  adequate  automatic  recording 


meohanlsm  there  are  few  recording  error*  or  misses  euch  aa  may  occur 
when  an  observer  haa  the  responsibility  for  recording  every  step  that 
the  subjsct  makes  in  the  course  of  hia  trouble  shooting. 

A further  advantage  la  that  the  response  records  are  presented 
in  a standardised  manner  - the  machines  do  not  adopt  idiosyncrasies 
of  style  which  consequently  make  subsequent  analysis  of  the  data  dif- 
ficult. finally,  there  is  an  advantage  of  shorter  administration  time. 
The  use  of  the  machines  enable  the  men  to  work  at  a faster  rate  than 
that  possible  on  the  operating  equipments,  so  each  man  oan  attempt 
more  problems  during  a standard  testing  session, 

XI.  DESCRIPTIQ1T  Of  TEJ  AUTCllASfS  HACHIHE 
The  Outside  of  the  Machine 

An  exterior  view  of  the  AUTOISASTS  testing  machine  is  shown  in 
figure  1.  The  machine  is  contained  in  a plywood  box  (approximately 
21"  x 25"  x 6")  with  ei  metal  lid.  The  entire  assembly  is  mounted  so 
that  the  plane  of  the  lid  makes  an  angle  of  20  degrees  with  the  table 
top.  This  arrangement  enables  the  man  working  with  the  equipment  to 
read  the  problem  information  as  it  is  exposed,  without  assuming  an 
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uncomf ortable  position.  A crank  handle  ( 1 - X)  extends  from  the 
right  side  of  the  machine.  A cast  iron  handwheel  (1  - f)  is  attached 
to  a vertical  shaft  which  protrudes  through  the  center  of  the  lid. 

The  tapered  information  slot  ( I - L)  at  the  right  of  the  handwheel  is 

jj 

Throughout  this  section  of  the  report  parenthesized  notations 
refer  to  the  figure  detail's,  for  example,  (I  - K)  indicates  that  the 
crank  handle  is  labeled  "X"  in  figure  X. 


( labeled  by  naans  of  a plastic  strip  of  "action"  labels  (1  - J) 

cemented  to  the  lid  above  the  slot,  The  slide-pointer  (I  - M) 
or  "action  selector"  may  be  set  in  any  one  of  the  five  labeled 
positions.  Eed  and  green  effect  indicator  lights  (1  - C)  are 
locatod  at  the  upper  left-hand  corner  of  the  lid.  These  lights 
are  also  identified  by  plastic  labels  (1  - D)  cemented  to  the 
lid.  She  lid  is  continously  hinged  to  the  box.  Slide  fasteners 
(I  — G)  located  at  the  right  and  left  edges  of  the  lid  fasten  the 
lid  firmly  when  it  is  in  the  closed  position*  A stiffening  brace 
(I  - B)  supports  the  lid  near  its  center.  The  rubber-padded  brack- 
et (I  - A)  on  the  outside  back  of  the  box  serves  as  a rest  for  the 
lid  when  the  box  is  open.  A plastic  knob  (I  - X)  and  pointer  are 
affixed  to  a rotating  shaft  which  protrudes  from  the  left  side  of 
f the  machine  and  a scale  (Z  - H) , calibrated  to  time  in  minutes,  is 

fastened  to  the  side  of  the  box  beneath  the  pointer  tip. 

An  arrangement  of  shutters  (III  - D)  is  interposed  between 
the  lid  of  the  machine  and  the  problem  information  so  that  the  ex- 
perimental subject  can  obtain  problem  information  by  following 
specified  procedures,  Vhen  these  procedures  are  followed,  a por- 
tion of  the  shutter  opens  (III  - F)  and  problem  information  can  be 
read  through  the  aperture.  The  portion  of  the  information  slot  to 
the  right  of  the  shutter  (III  - G & H)  is  always  open  end  allows 
the  subject  to  see  the  position  of  the  problem  information  disk. 

The  the  Machine  is  Used 

General  procedures  for  the  administration  of  the  AUTOIIASTS 
( test  will  be  presented  in  Section  IV  of  this  report,  but  a brief 
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Timer  scale 


figure  I 


description  of  the  tank  of  the  experimental  subject  is  given  here 
in  order  to  facilitate  the  further  description  of  the  machine* 

Bach  experimental  subject  is  provided  with  an  AUTOMASTS  test- 
ing machine,  a set  of  schematic  diagrams,-*  and  other  reference 
material.  At  the  boginning  of  a problem  he  is  handed  a symptom 
card  bearing  the  output  characteristics  of  the  electronic  equip- 
ment upon  which  the  problem  is  based.  Tor  example , if  the  problems 
sore  based  upon  a radio  receiver,  the  legend  on  the  symptom  card 
might  read  "no  sound  at  any  station,"  Shis  means  that  the  set  can- 
not receive  on  any  frequency.  The  task  of  the  experimental  subject 
is  to  find  and  correct  the  difficulty.  Be  does  this  by  sampling  a 
large  number  of  test  equipment  readings  obtained  at  various  points 

6 

in  the  circuit  or  by  determining  the  effects  of  substituting  parts. 

In  its  present  form  the  machine  permits  him  to  determine  the 

results  of  applying  four  types  of  test  equipment  to  the  radio  re- 

7 

ceivor  circuit.  These  are  I a signal  generator,  an  AC  voltmeter, 
a BO  voltmeter,  and  an  ofammeter.  Tour  of  the  five  plaBtic  labels 
above  the  tapered  slot  on  the  lid  correspond  to  these  test  equip- 

“ k 

•'Copies  of  these  schematic  diagrams  appear  in  the  Appendix. 

^In  many  ways  the  testing  situation  resembles  the  situation 
which  would  exist  if  the  person  attempting  to  solve  the  problem 
were  cn  one  side  of  a black  curtain  with  the  malfunctioning  gear, 
a set  of  teBt  equipment,  and  a trained  technician  on  the  other.  In 
effect,  the  man  attempting  to  Bolve  the  problem  asks  the  unseen 
technician  to  make  certain  checks  at  particular  points  in  the  cir- 
cuit and  to  report  the  results, 

^Tor  uulse-f orming  circuits  an  oscilloscope  is  substituted  for 
the  signal  generator. 
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Key  for  Pigure  XI 


A*  Hols  for  center  shaft 
B.  Hinge 

0*  Information  slot  4 

B.  Effect  indicator  light  sockets 
E,  Belay  system  micro-switch 
7.  Belay  system  relay 

G.  Belay  timer 


H.  Center  shaft 

I.  Shutter  support  plate 

J.  Problem-information  disk 

X.  Problem-information  disk  indexing 
hole  and  pin 
L.  Printing  disk 
M*  Indexing  disk 


It  Positioning  bearing 

O,  Locking  timer 

P,  Supply  spool 

Q,  Take-up  spool 

B.  Paper  tape  (showing  printed  record 
of  trouble  shooting  steps) 

8,  Printer  holding  bracket 


?•  Pointing  unit 
U«  Paper  advanoe  knob 
V.  Mechanical  linkage  - 
V.  Shutter  support  poet 

X,  Shutter  advance  bar 

Y,  Bhutter 

Z,  Slide-pointer  stud 
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ments.  The  fifth  label  reads,  "Replace  or  Adjust  Component. " When 
the  slide-pointer  under  the  Blot  in  the  lid  is  positioned  to  one  of 
the  action  labels  (such  as  D,  0.  Volte)  and  the  crank  handle  is 
pulled,  the  shutter  opens  in  the  information  slot  and  reveals  the 
information  which  would  have  been  obtained  if  a DO  Voltmeter  had 
been  applied  to  the  actual  electronic  gear  under  the  circumstances 
of  the  problem.  When  the  si idfr*po inter  is  directed  toward  the 
"Replace  or  Adjust  Component"  position  (and  the  crank  handle  is 
pulled)  the  subject  receives  the  results  of  substituting  a good 
tubs,  resistor,  capacitor,  or  lead  for  one  already  in  the  circuit-. 
The  "Replace  or  Adjust  Component"  position  may  also  be  used  to  de- 
termine the  effects  of  adjusting  components,  such  as  a tuneable 
oondenser,  which  are  suspected  to  bo  the  cause  of  the  trouble* 

The  results  of  the  action  of  replacing  or  adjusting  components 
are  given  by  a pair  of  effect  indicator  lights  in  the  upper  left- 
hand  corner  of  the  lid.  If  the  component  replaced  is  the  cause  of 
the  faulty  output  of  the  equipment,  a gr^en  light  labeled  "gear 
normal"  v/ill  come  on.  If  the  component  replaced  is  not  the  cause 
of  the  symptoms,  a red  light  labeled  "no  effect"  will  indicate  that 
the  set  is  still  not  functioning  properly. 

The  experimental  subject  may  make  any  of  the  four  types  of 
reading  at  any  of  120  points  in  the  circuit.  Similarly,  he  may  r»» 
place  any  one  of  120  components.  All  the  possible  test  points  and 
all  the  replaceable  components  are  numbered  on  the  schematic  dia- 
gram. To  make  a check  at  some  particular  point  in  the  circuit, 
the  subject  rotates  the  center  handwheel  of  the  machine,  in  either 
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Key  for  Figure  I IX 


1 


A,  Stiffening  brace 

B,  Action  labels 

0,  Center  handwheel 
I).  Shutter 

S,  Traverse  slot  for  slide-pointer  stud 

J,  Scope  reading  test  point  92  viewed 
through  open  window  in  shutter 

0.  Component,  number 
B.  Test  point  number 

1.  Slide-pointer 

J.  Slide-pointer  knob 


Key  for  figure  IV 

A sector  of  a problem- inf ormat ion  disk  for  the  radar  circuit  is 
shown  in  this  figure;  the  complete  disk  is  about  18  inches  in  diameter. 
The  information  is  arranged  in  a series  of  concentric  rings.  Starting 
from  the  outside  ring  and  working  toward  the  center,  the  information 
rings  are  arranged  in  the  following  order; 

Test  point  numbers  (89  through  95  shown) 
Component  numbers  (289  through  295  shown) 
Waveforms,  obtained  on  a test  oscilloscope 
DC  volt  readings 
AO  volt  readings 
Ohms  readings 

The  innermost  ring  shown  corresponds  to  the  "Replace  or  Adjust 
Component"  position  of  the  slide  pointer.  Ho  entries  appear  in  this 
ring  since  the  results  of  making  replacements  and  adjustments  are  pre- 
sented to  the  experimental  subject  by  the  effect  indicator  light  sys- 
tem, It  will  be  noted  that  the  sector  shown  in  Figure  IY  includes  the 
reading  which  is  shown  in  Figure  III, 
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FIGURED!.  FRONT  PANEL  DETAIL. 


FIGURE  32  PROBLEM  INFORMATION 
DISK  DETAIL. 


direction,  until  the  number  which  corresponds  to  the  point  on  the 

schematic  comes  into  view  in  the  space  to  the  right  of  the  shutter. 

He  sets  the  slide-pointer  to  indicate  the  type  of  check  that  he 

S 

wishes  to  make,  and  then  he  pulls  the  crank  handle. 

The  Center  Assembly 

The  center  handwheel  of  the  machine  is  locked  to  the  center 
shaft;  the  center  shaft  rotates  in  a ball  bearing  plate  which  ie 
attached  to  the  bottom  of  the  box.  When  the  subject  turns  the 
handwheel,  the  entire  shaft  rotates.  Three  large  disks,  each  approx- 
imately IS"  in  diameter,  are  mounted  on  the  shaft  and  turn  with  it. 
The  indexing  disk  (II  - M)  is  made  of  l/S*  steel  and  is  mounted 
closest  to  the  bottom  of  the  box.  One  hundred  and  twenty  wedge- 
shaped  notches  are  milled  into  its  circumference  on  3 degree  cen- 
ters, When  the  crank  handle  is  pulled  a wedge-shaped  dog  fits  in- 
to a notch  so  that  the  center  shaft  is  locked  at  one  position. 

When  the  crank  handle  is  returned  to  its  normal  position,  the  dog 
is  retracted  and  the  center  shaft  nay  be  rotated  again.  By  means 
of  spring  tension,  a small  positioning  bearing  (II  - IT)  is  held  in 
contact  with  the  edge  of  the  indexing  disk.  Thi3  bearing  assists 
in  the  gross  positioning  of  the  indexing  disk  and  also  masks 
slight  mechanical  noises  that  might  serve  as  cues  to  the  solution 
of  the  problen  if  masking  were  not  employed. 

The  printing  disk  (II  - L)  is  no-anted  on  the  center  shaft  par- 

Figure  III  shows  the  wave  form  on  the  test  oscilloscope  when 
it  is  connected  between  test  point  number  $?.  and  ground. 
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allel  to  and  about  one  inch  above  the  indexing  disk.  The  printing  disk 
is  made  from  }/&*  aluminum  plate.  Rectangular  slots  are  milled  in  the 
edge  of  the  disk  on  3 degree  centers  to  match  the  indexing  notches  in 
the  steel  indexing  disk.  Lead  type  slugB  are  cemented  into  the  slots 
in  the  printing  disk  and  protrude  slightly  beyond  the  circumference; 
the  type  slugs  are  arranged  in  consecutive  order  with  numbers  running 
from  1 through  120, 

A thin  aluminum  problem" inf ormation  disk  (IX  - J)  is  situated  li 
inches  above  the  printing  disk.  Information  for  each  problem  is  ar- 
ranged in  concentric  rings  on  a paper  cemented  to  the  aluminum  disk 
(see  Figure  IV).  Nevr  problems,  i.e.t  troubles,  are  introduced  into 
the  machine  by  substituting  new  problem- information  diBks.  A small 
Indexing  hole  (II  - K)  in  the  problem- informat  ion  disk  fits  over  an 
indexing  pin  on  the  printing  disk  so  the  problem  information  is  posi- 
tively aligned  with  both  the  printing  disk  and  the  indexing  disk. 

When  the  experimental  subject  turns  the  handwheel,  the  entire  center 
assembly  (center  shaft,  indexing  disk,  printing  disk,  and  problem- 
information  disk)  is  rotated.  The  rotation  of  this  assembly  brings 
the  proper  test  point  number  under  the  open  portion  of  the  information 
slot  in  the  lid  of  the  machine.  This  action  also  positions  the  print- 
ing disk  so  that  the  number  printed  on  the  paper  tape  (II  - R)  cor- 
responds to  the  number  of  the  test  point  in  the  slot* 

A heavy  metal  plate  (II  - I)  fits  over  the  center  shaft  and  rests 
on  the  problem- informat  ion  disk;  this  plate  serves  as  a bearing  surface 
for  the  free  edge  of  the  shutter. 

The  cast  metal  handwheel  is  fitted  to  the  center  shaft  with  a 
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retaining  pin.  Both  handwheel  and  pin  are  removable  so  that  the 
box  lid  may  be  opened  to  change  the  problem- information  disks  and 
to  service  the  machine. 

The  Shutter  Assembly 

The  position  of  the  front  panol  slide-pointer  is  reoorded  on 
the  paper  tape  by  alphabetical  symbols.  Thus,  nAK  indicates  a 
signal  generator  or  oscilloscope  reading,  "B*  indicates  DC  volts, 
etc.  Thcco  symbols  are  printed  by  type  slugs  mounted  on  the  shut- 
ter-advance bar  (II  - X).  Tliic  bar  slides  on  grooved  studs  which 
are  fastened  to  the  bottom  of  the  box.  When  the  crank  handle  is 
pulled,  the  shutter-advance  bar  is  slippod  along  itB  mounting 
grooves  by  a system  of  mechanical  linkages  (II  - V).  The  extent 

of  this  movement  is  controlled  by  the  indexing  Blots  in  the 

9 

shutter. 

Holes  in  the  shutter  slip  ov  ir  pins  on  the  top  of  the  shut- 
ter-support posts  (II  - W)  to  rigidly  position  the  shutter  with 
respect  to  the  slide-pointer  stud;  spring  clips  hold  the  shutter 
on  the  pins.  The  shutter  displays  information  on  tho  problem- 
information  disk  by  means  of  five  staggered  apertures  which  are 
located  so  that  exactly  one  item  of  information  can  be  viewed 
through  the  slot  in  the  lid  of  the  box  at  a tirao.  Thus,  if  the 
subject  turns  the  wheel  to  test  point  j6  and  sots  the  slide 

^ Those  slots  are  shown  but  not  labeled  in  Figure  II.  They 
consist  of  five  vertical  slots  connected  at  their  top  by  a hori- 
zontal slot.  The  vortical  slots  differ  in  length  so  as  to  prop- 
erly position  the  shutter  apertures  and  the  shutter  advance  bar. 
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pointer  to  DC  volte,  only  the  DC  volte  reeling  will  appear  in  the 
window  when  the  handle  ie  pulled.  Of  course,  the  subject  can  take 
any  other  type  of  reading  at  the  sane  test  point,  by  resetting  the 
slide-pointer  and  pulling  the  crank  handle,  but  each  reading  appears 
separately  and  ie  printed  separately. 

The  shut  ter- indexing  slots  are  situated  toward  the  lower  end  of 
the  shutter.  Vhen  the  lid  of  the  box  ie  closed  and  the  orank  handle 
is  in  the  normal  position,  the  slide-pointer  stud  (11  - Z)  engages 
the  horizontal  slot.  When  the  crank  handle  is  pulled,  the  slide-pointer 
stud  restricts  the  movement  of  the  shutter  in  the  following  ways) 

First,  if  the  slide-pointer  is,  not  pointing  directly  at  some  one  labeled 
location  in  the  information  slot,  the  shutter  will  not  advance  and 
no  information  will  be- disclosed.  Second,  if  the  slide-pointer  is 
pointing  directly  at  some  one  labeled  location  in  the  slot,  the  shut- 
ter ie  free  to  move  relative  to  the  slide-pointer  stud  until  the  shut- 
ter comes  into  contact  with  it.  When  the  shutter  is  stopped  by  the 
stud,  the  appropriate  aperture  is  properly  positioned  in  the  informa- 
tion slot  so  the  subject  may  read  the  desired  problem  information 
from  the  disk,  and  the  type  of  action  taken  iB  printed  on  the  paper 
tape  by  means  of  the  type  slugs  on  the  shutter  advance  bar. 

The  Printing  Unit 

The  printing  unit  consists  of  a rubber  platen,  a typewriter  rib- 
bon drive  system,  and  a paper  advancing  device.  The  unit  is  suspended 
between  a socket  in  the  bottom  ox"  the  box  aud  the  printer-holding 
bracket  (XI  - S) . When  the  crank  handle  iB  pulled,  the  printer  ie 
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pivoted,  by  cam  and  roller  action,  around  the  axis  of  its  support- 
ing post.  This  movement  forces  the  platen  firmly  against  both  the 
type  that  extends  from  the  circumference  of  the  printing  disk  and 
the  type  that  is  mounted  on  the  shutter  advance  bar.  Vhen  the 
crank  handle  is  returned  to  normal  position,  the  printer  swings 
cloar  of  the  type  and  the  paper  tape  is  advanced  by  a ratchet* 
During  this  return  stroke,  the  movement  of  the  paper  drives  a fric- 
tion wheel  which  advances  the  typewriter  ribbon  through  a system 
of  bevel  gears.  When  necessary,  the  ribbon  drive  gears  are  re- 
versed by  pressing  the  end  of  the  ribbon-reversal  shaft.  Thus, 
for  each  pull  of  the  crank,  a double  entry  is  printed  on  the  tape 
such  as  "A  Ig),"  "D  76,*  or  *E  36."  the  paper  advances  one-half 
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inch,  and  the  typewriter  ribbon  advances  one-quarter  of  an  inch. 
Effect  Indieat ing  System 

A small  plastic-covered  bolt  projects  from  the  underside  of 
each  problem-information  disk.  This  bolt  makes  contact  with  the 
blade  of  a mercury  switch  which  is  mounted  on  the  inside-back  of 
the  box.  When  the  problem- inf ormat ion  disk  is  positioned  so  the 
number  of  the  f >.lty  component  may  be  seen  through  the  information 
slot  in  the  lid  of  the  box,  the  blade  of  the  mercury  switch  is 
depressed.  If  the  switch  blade  is  depressed  and  if  the  slide- 

10 

The  printed  letters  refer  to  the  type  of  reading  and  the 
printed  numbers  refer  to  the  test  point  numbers.  The  printed  entry 
"A  40N  means  that  a signal  was  injected  at  test  point 
means  that  resistance  to  ground  was  read  at  test  point  "E  36" 
indicates  that  the  cathode  follower  tube  in  the  radar  circuit  was 
replaced. 
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pointer  is  in  the  "Replace  or  Adjust  Component"  position,  the  green  "Gear 
Normal"  effect  indicator  light  comes  on  }0  seconds  after  the  crank  handle 
is  pulled.  If  the  blade  of  the  mercury  switch  is  not  depressed,  hut  the 
slide-pointer  is  in  the  appropriate  position,  the  red  "No  Effect"  indica- 
tor light  will  come  on  at  the  end  of  the  delay  interval. 

This  30-eecon<i  interval  1b  controlled  by  a timer  through  a system  of 
switches  and  a relay.  The  delay  was  introduced  to  minimize  the  likelihood 
of  chance  success  and  to  serve  as  a deterrent  to  excessive  parts  replace- 
ment. The  timer  used  could  he  varied  from  0 to  60  seconds.  Tryouts 
revealed  that  a 30-second  delay  was  sufficient  to  discourage  aimless 
replacing  activity  on  the  part  of  the  subject,  and  still  did  not  use  up 
an  inordinate  amount  of  the  time  allowed  for  the  diagnosis  of  each  prob- 
lem. 

Out  Off  Timer 

Another  timer  i3  located  in  the  lower  left-hand  corner  of  the  box. 
This  timer  serves  two  functions.  It  indicates  the  elapsed  time  for  each 
problem  by  means  of  a pointer  and  scale.  It  also  activates  an  electrical 
switch  at  the  end  of  a specified  interval.  This  switch  operates  a lock- 
ing relay  which  engages  a projection  on  the  cam  so  that,  the  crank  handle 
can  no  longer  bo  pulled.  In  this  manner,  the  machine  may  be  set  to  auto- 
matically lock  at  the  problem  time  limit  or  at  the  end  of  any  prescribed 
period  of  time  from  0 to  60  minutes. 

Consideration  vaB  given  to  several  methods  for  marking  the  paper 
tape  with  a time  base.  Provision  was  made  to  add  a simple  time  indi- 
cating device  to  the  printing  mechanism  but  the  actual  mechanism  will 
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not  be  add'd  until  tha  advantages  of  having  the  tine  base  have  been 
shown  to  warrant  the  additional  expense. 

Summary  of  the  Description  of  the  Machine 

To  summarize  the  description  of  the  AUTCMASTS  tenting  machinal 
It  is  essentially  a box  with  a crank  handle  on  the  right  side  and 
a handwheel  . in  the  center  of  the  lid.  The  experimental  subject 
may  turn  the  handwheel  to  bring  any  one  of  120  test  point  numbers 
into  view  through  a slot.  A slide-pointer  below  the  slot  may  be 
set  to  one  of  several  positions;  each  position  indicates  a certain 
type  of  check  that  is  made  in  the  circuit.  Vfhen  the  crank  handle 
is  pulled  a 6mall  window  opens  and  revoalB  the  information  that 
v;ould  be  obtained  by  use  of  the  indicated  test  equipment  under  the 
circumstances  of  the  problem.  The  slide-pointer  and  handwheel  can 
also  be  manipulated  to  determine  the  effects  of  replacing  or  tuning 
certain  parts  in  the  circuits.  Every  check  or  replacement  that  the 
experimental  subject  makes  is  automatically  recorded  in  coded  fora 
on  a paper  tape.  Accessory  devices  introduce  delays  into  the  re- 
placement cycle,  indicate  elapsed  time,  and  lock  the  crank  handle 
at  the  end  of  a prescribed  period, 

III.  AUTCliASTS  TROUBLE  SHOOT I1TG  PROBLEMS 

The  Basic  Electronic  Equipment 

The  problems  employed  in  the  present  form  of  the  AUTOliASTS 
were  selected  from  those  pretested  in  the  MASTS  format.  They  were 
originally  developed  by  introducing  malfunctions  into  functioning 


-23- 


electronic  equipments.  Those  equipments  were  a superheterodyne  radio 
receiver  and  a radar  sweep  generator  assembled  from  a training  kit* 
manufactured  by  the  Philco  Corporation.  Several  different  circuit* 
can  be  assembled  from  the  available  chassis.  The  selection  of  cir- 
cuits  for  this  test  development  was  governed  by  the  desire  to  employ 
circuits  simple  enough  to  permit  relatively  rapid  subject  orientation 
and  yet  complex  enough  to  require  realistic  trouble  shooting  behav- 
ior. The  fact  that  the  circuits  chosen  were  of  a common  sort  and 
yet  not  exactly  the  some  as  any  gear  the  men  would  be  expected  to 
have  come  in  contact  with  reduced  differential  effects  of  special- 
ization on  certain  gears  and  still  permitted  the  operation  of  gen- 
eral experiential  factors. 

The  Bases  of  Problem  Selection 

Twelve  problems  (i,e. , trouble  shooting  tasks)  were  selected 
from  the  original  30  developed  for  MASTS  use.  The  principal  criter- 
ion for  problem  selection  was  the  correlation  between  success  on 
the  problem  when  given  in  the  MASTS  format  and  success  in  locating 
and  repairing  troubles  in  the  actual  equipments.  Care  was  also 
taken  to  choose  problems  which  were  representative  of  different 
stages  of  the  equipment,  different  types  of  faulty  components,  and 
a variety  of  symptoms.  The  faults  selected  were  typical  of  those 
which  normally  occur  in  such  equipment. 

*”*  * This  kit  consists  of  many  individual  stage  chassis.  For 

example,  the  radio  power  supply  is  a separate  stage,  the  R-F  ampli- 
fier is  another  stage,  and  so  on.  Each  chassis  has  plugs  for  cou- 
pling to  other  stages  and  for  making  ground  and  power  connections. 


-24- 


r 


Description  of  the  Problems 

Tho  problem  information  presented  on  the  problem- information 
disks  was  obtained  by  taking  readings  between  each  of  the  numbered 
tost  points  and  ground  with  each  type  of  test  equipment  under  the 
circumstances  of  each  problem. 

Twelve  AUTOIIASTS  problems  is  the  maximum  feasible  number  for  a 
half  day  of  group  testing.  Although  a longer  test  would  have  im- 
portant psychometric  advantages,  it  could  not  be  conducted  within 

12 

realistic  time  limits. 

Pour  types  of  electronic  circuit  components  provide  the  loci 
for  the  AUTOIIASTS  problems.  Table  1 shows  the  tubes,  resistors, 


Table  1 

Types  of  Components  Serving  as  Loci  for  Troubles  in 
Problems  Selected  for  AUTOIIASTS  Administration 


Type  of  Component* 

Humber  of  Times  Represented  in* 

Radio  Circuit 

Radar  Circuit 

Tube 

r 

i i 

1 

Resistor 

s ; 

i 

3 

Capacitor 

2 ! 

1 

Lead 

1 

1 

1 

Each  trouble  consists  of  or.e  faulty  component,  al- 
though a casualty  may  reflect  throughout  the  circuit. 


I 


TIT 

However,  the  AUTOiiASTS  format  permits  the  administration  of 
more  problems  per  unit  time  than  the  ,job  cample  test  upon  which  it 
is  based.  The  maximum  time  limit  for  twelve  problems  in  AUTOiiASTS 
form  (less  instructions)  is  P-l/2  hours.  The  Bame  problems  in  the 
equipment  require  a maximum  timo  limit  of  7 hours. 
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capacitors,  and  leads  used  in  each  circuit.  No  troubles  are  located 
in  the  radio  speaker  or  the  cathode  ray  tube  of  the  radar  sweep  genera- 
tor circuit.  Trancformer  troubles  are  indistinguishable  from  troubles 
located  in  the  leads  to  the  transformer.  Since  defective  leads  are 
more  easily  obtained  than  defective  transformers,  the  faulty  leads  are 
used. 

Pour  types  of  defects  were  incorporated  in  the  AUTCtlASTS  problems, 
A component  could  be  open,  shorted,  detuned,  or  of  the  wrong  value. 
Table  2 shows  the  distribution  of  these  defects  in  the  two  circuits. 

All  of  the  defective  components  selected  for  use  produced  a notice- 
able effect  upon  the  output  of  the  prototype  equipments,^ 

Table  2 

Nature  of  Defects  Represented  in  Problems  Selected 
for  AUTdlASTS  Administration 


Number  of  Times  Represented  ini 

Nature  of  Defect  Radio  Circuit 

Open  component  3 

3 

Shorted  componert  1 

2 

Detuned  component  1 

0 

Wrong  value  com-  1 

1 

ponent 

— - 

Normal  output  for  the  receiver  was  faithful  reproduction  At 
the  speaker  of  signals  received  within  the  commerical  broadcast  band. 
Normal  output  for  the  radar  sweep  circuit  consisted  of  a number  of 
vertical  pips  equally  spaced  along  the  horizontal  sweep  of  the  moni- 
tor scope. 
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While  it  is  not  possible  to  represent  all  of  the  stages  in  the 
circuits  with  such  a small  number  of  problems « efforts  were  made  to 
select  problems  which  were  distributed  throughout  the  equipment 
(i. e. , not  all  in  the  Bame  stage).  The  locations  of  the  defective 
components  employed  are  given  in  Tables  3 and  4, 

Table  3 

Radio  Receiver  Stages  in  which  Defective 
Component a wore  Located 


Stage 

dumber  of  Problems  in  which 
Defective  Component  was  in 
this  St&ffe 

Audio  voltage  am- 

2 

pi  if  ier 

Detector 

2 

I-F  amplifier 

1 

R-P  converter 

1 

Table  4 

Radar  Sweep  Generator  Circuit  Stages  in  which 
Defective  Components  were  Located 


Stage 

Rummer  of  Problems 
in  which  Defoctive 
Component  was  in 
this  Stage 

Single  swing  blocking  oscil- 
lator 

1 

Trigger  blocking  oscillator 

1 

One  shot  multivibrator 

2 

Sawtooth  genorator 

1 

Monitor  oscilloscope 

1 

The  defective  components  which  produoe  the  "troubles"  for  the 
AUT01IASTS  problems  represent  different  types  of  electronic  functions 
within  the  normally  operating  equipment.  Tor  example,  one  of  the  two 
defective  radio  capacitors  acts  as  a by-paes  capacitor  while  the  other 
is  a tuneable  element  in  a tank  oirouit. 

The  symptoms  for  the  malfunctions  consist  of  the  output  charac- 
teristics of  the  equipments  when  the  faulty  part  is  inserted.  These 
symptoms  are  depicted  for  the  subject  on  a small  oard  exposed  at  the 
beginning  of  each  problem.  Three  different  symptoms  are  used  for  the 
radio  problems.  They  were!  "no  sound  at  any  station*  (indicating 
that  no  sound  was  produced  at  the  loudspeaker  even  though  the  equip- 
ment was  turned  on  and  the  tuning  dial  was  adjusted  to  the  frequency 
of  a coaaerical  broadcast  station),  "weak  signal  at  all  stations," 
and  "very  weak  signal  at  all  stations,”  live  different  radar  symp- 
toms were  represented.  The  symptom  card  showed  a diagram  of  the  face 
of  the  monitor  scope.  The  symptoms  portrayed  the  base  line  without 
vertical  pips,  with  irregularly  spaced  pip a without  a horisontal  sweep, 
and  in  one  problem  the  entire  pattern  on  the  face  of  the  scope  was 
off  centered. 


rr.  ASHXNXSTBAf ION  or  THB  AUTOHASTS  TS3T 

General 

The  AUTCl LASTS  is  especially  designed  for  group  administration, 
and  the  number  of  subjects  that  oan  be  tested  at  one  time  is  limited 
principally  by  the  number  of  machines  and  the  amount  of  space  avail-' 
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able.  The  present  administrative  procedures  have  been  applied  in 
the  field  to  groups  of  five  subjects  at  a time;  probably  no  major 
changes  in  procedure  would  be  necessary  for  administering  the 
AUT011ASTS  to  larger  groups. 

Situational  Requirements 

Sach  subject  is  provided  with  an  AUTOKASTS  machine,  a sche- 
matic, and  a reference  manual  for  the  equipment  he  is  trouble  shoot- 
ing. For  each  man,  a desk-height  working  surface  of  about  3x4 
feet  is  required.  It  is  desirable  to  provide  individual  desks  or 
tables  bo  that  each  man  can  work  independently.  Most  subjects  like 
to  alternate  between  standing  and  sitting  as  they  woik;  the 
AUTOilASTS  can  be  operated  from  either  position.  Ordinary  class- 
room lighting  is  sufficient  for  AUTGliASTS  testing  as  the  instru- 
ment makes  no  special  visual  demands  on  the  subject.  Sach  machine 
must  be  connected  to  an  electrical  outlet  to  provide  current  for 
the  cutoff  timer  and  effect  indicator  light  systems. 

Tine  Requirements 

The  present  AUTCLASTS  Test  consists  of  twelve  problems,  six 
with  a ten-minute  cutoff  and  six  with  a fifteen-minute  cutoff.  The 
average  group  administration  time  is  about  three  and  one-half  hours. 
This  time  allows  for  slower  subjects  who  take  almost  the  full  time 
limit  on  every  problem.  Of  course,  if  the  group  ie  composed  of  fast 
trouble  shooters  the  test  c?.n  be  administered  in  a shorter  time. 

Testiiy;  Procedures 

Before  the  tost  problems  are  introduced  each  subject  partici- 


pats*  in  a training  session  to  acquaint  him  with  the  testing  procedures 
and  the  circuits  with  which  he  is  to  work.  The  bulk  of  the  instruc- 
tion is  presented  in  the  form  of  a tapo-recorded  lecture.  This  pro- 
cedure frees  the  examiner  so  that  he  can  clarify  and  amplify  parts  of 
the  instructions,  if  necessary.  Pauses  during  the  tap»*recorded  in- 
structions permit  the  examiner  to  demonstrate  features  of  the  AUTOMA.STS 
machine.  While  the  record  is  being  played  he  obseryas  each  subject  to 
be  sure  thut  the  instructions  are  being  carried  out  properly. 

Typical  Testing  session 

Radio  orientation.  In  the  typical  testing  session  the  radio  re- 
ceiver problems  are  given  first.  During  the  training  period  the  recei- 
ver is  described  carefully,  and  each  subject  notes  the  different  stages 
and  signal  paths  on  his  own  schematic.  The  numbering  system  for  test 
points  and  components  is  made  explicit,  and  the  subjects  are  required 
to  show  that  they  understand  the  staging  and  numbering  system  by  finding 
several  standardised  check  points. 

Next,  the  subjects  are  shown  how  to  set  the  machine  for  the  dif- 
ferent test  points  and  now  to  adjust  the  slide-pointer  to  give  the  dif- 
ferent types  of  readings.  The  features  of  the  machine  which  prevent 
the  subject  from  taking  more  than  one  reading  at  a time  are  demonstrated. 

M the  conclusion  of  the  instructions  covering  the  use  of  the  sig- 
nal generator  a five  minute  period  is  devoted  to  taking  practice  read- 
ings on  the  set  of  normal  receiver  readings  in  the  machine.  During 
this  time,  the  subjects  are  encouraged  to  consult  their  equipment  manu- 
als and  verify  some  of  the  normal  voltage  readings  and  parts  specifica- 
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tions  that  are  given  there*  The  five  minute  period  has  been  found 
to  be  adequate  for  the  taking  of  practice  readings  on  the  present 
equipment;  longer  periods  might  be  necessary  if  more  console*  equip- 
ments were  adapted  to  the  AUTOliASTS  format. 

The  effect  indicator  light  and  delay  systems  are  graphically 
presented  to  the  subjects  by  having  them  "replace"  a faulty  com- 
p<  at  while  the  normal  information  disk  is  in  the  machine.  The 
normal  disk  has  been  equipped  so  that  replacing  one  of  the  parts 
will  give  a "gear  normal*  indication;  each  subject  is  told  to  re- 
place this  part  and  observe  the  green  light,  then  to  replace  any 
other  part  and  observe  the  red  or  "no  effect"  light* 

The  instructions  are  very  thorough.  It  takes  almost  half  an 
hour  to  bring  technicians  to  the  point  where  they  are  ready  for 
a practice  trouble  shooting  problem.  Field  experience  indicates 
that  the  time  spent  on  thorough  instruction  is  well  justified. 

Radio  practice  problem.  Aft^r  the  radio  orientation  pro- 
cedures, each  subject  is  given  a radio  familiarization  problem.  He 
is  presented  with  a symptom  card  and  told  to  trouble  shoot  the  equip- 
ment by  taking  readings  and  replacing  parts.  During  the  practice 
problem,  the  examiner  answers  any  questions  that  the  subject  may 
have  about  the  operation  of  the  machine  or  the  general  charac- 
teristics of  the  electronics  equipment. 

The  practice  problem  is  rather  easy  and  most  sujects  solve  it 
before  the  ten  minute  time  limit.  Unsuccessful  subjects  are  given 
the  answer  to  the  practice  problem  before  proceeding,  and  they  have 
an  opportunity  to  take  a few  readings  to  "verify"  the  answer. 


Radio  test  problems.  For  each  toot  problem,  a now  problem- informa* 
tlon  disk  i.j  Inserted  in  the  machine,  the  paper  tape  is  advanced  several 
Inches,  and  the  technician  Is  given  a new  symptom  card  which  describes 
the  gross  output  symptoms  of  the  aotual  equipment.  The  subject  works 
alone  during  the  trouble  shooting  task.  No  questions  or  interruptions 
are  permitted  during  the  test  problems,  Zf  the  teohnician  questions 
the  validity  of  particular  information  that  he  gets  from  the  machine, 
he  is  told  that  the  problem  information  has  been  checked  and  standard* 
ised,  and  that  he  will  have  a later  opportunity  to  discuss  special 
points. 

A total  of  six  radio  problems  are  administered  with  a time  limit 
of  ten  minutes  per  problem.  All  subjeots  take  the  same  problems  in 
the  same  order,  Xf  a subject  fails  to  solve  a problem  within  the  time 
limit,  the  machine  looks  and  he  is  informed  of  the  correct  answer  be- 
fore going  on  to  the  next  problem. 

Badar  orientation.  Nhen  all  subjects  have  finished  the  radio  prob- 
lems, the  radar  problem- information  disk  showing  the  readings  obtained 
when  the  radar  equipment  was  functioning  normally  is  placed  in  the 
machine.  A tape-recorded  radar  orientation  lecture  is  played  to  the 
group.  The  radar  instructions  are  similar  to  the  radio  instructions 
in  that  each  subject  is  led  through  the  scheaat ic  and  is  required  to 
take  standardised  readings  from  the  machine.  However,  the  radar  orien- 
tation is  accomplished  in  only  15  minutes  because  the  subjects  are 
already  familiar  with  the  operating  features  of  the  testing  machine. 

Badar  practice  problem.  At  the  conclusion  of  the  radar  orien- 
tation lecture  each  subject  works  a simple  radar  practice  problem. 
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This  procedure  permits  the  men  to  demonstrate  that  they  have  under- 
stood the  Instructions  and  have  achieved  a degree  of  familiarity 
with  the  circuit.  Almost  all  men  solve  the  practice  problem  within 
the  15  minute  time  limit.  The  examiner  works  with  any  subject 
having  difficulty  during  the  practice  problem. 

Badar  test  problems.  The  radar  problems  are  administered  in 
the  same  manner  as  the  radio  problems.  A new  problem- information 
disk  and  symptom  card  is  used  for  each  problem.  If  a technician 
does  not  solve  a problem  within  the  15  minute  cutoff  time*  he  is 
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given  the  correct  answer  before  going  on. 

Performance  records,  The  examiner  enters  the  time  required  by- 
the  subject  to  solve  each  problem  on  a standard  form.  At  the  end 
of  the  Complete  AUTOIIASTS  session,  the  printed  tape  is  removed  from- 
the  machine.  The  tape  of  trouble  shooting  responses  and  the  AUTO- 
MASTS time  records  constitute  the  basic  data  of  the  test. 

7,  PLANS  TOR  ANALYSIS  OP  DATA 

The  AUTOHASTS  data  constitute  a real  challenge  to  the  data 

15 

analyst  because  there  are  eo  many  possibilities  for  scoring  the 
r'm  ’ lir 

There  were  two  reasons  for  doing  this)  (l)  to  provide  olo- 
sure,  and  (2)  to  provide  a better  understanding  of  the  dynaaio 
features  of  the  circuit.  This  latter  was  to  reduce  the  advantages 
whioh  might  aocrue  to  those  who  Bolved  the  initial  problem  over 
those  who  did  not, 

^In  this  report,  the  term  "scoring"  refers  to  the  process  of 
abstracting  certain  quantifiable  features  from  the  behavioral 
records. 
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trouble  shooting  performance.  She  general  plan  adopted  here  le  to  develop 
ae  large  a variety  of  eooring  methods  ai  possible  and  to  apply  then  to 
data  collected  in  the  field.  An  eepirical  study  of  the  relationship! 
between  the  varloue  ecore  parameters  can  then  be  expected  to  lead  to 

(1)  a better  understanding  of  the  trouble  shooting  process,  (2)  bases 
for  combining  different  scores,  and  (3)  improved  predictive  properties 
of  the  AUTGIUSSS  test. 

She  discussion  below  mentions  the  types  of  sooring  procedures 
which  are  now  being  applied  %o  AUTCHASTS  data,  Many  of  these  have  been 
proposed  or  applied  by  other  investigators,  and  the  present  collection 
of  scoring  methods  will  undoubtedly  be  supplemented  and  modified  as 
the  analysis  prooeeds. 

Certain  end  products  are  conventionally  derived  from  performance 
test  records.  A testae  is  usually  given  credit  for  the  number  of  tasks 
he  successfully  performs  or  the  number  of  troubles  he  finds  under  the 
circumstances  of  the  test.  The  principal  difficulties  in  relying  sole- 
ly upon  this  score  are I (l)  practical  factors  necessitate  a relatively 
small  number  of  items  and  a consequently  restricted  range  of  scores  and 

(2)  the  end-product  score  ignores  all  the  prooess  information  contained 

in  the  response  records.  If  each  trouble  shooting  problem  is  regarded 
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as  an  item  in  a test,  the  discriminating  power  of  each  item  may  be 


1 '’Technical  Report  Ho.  9 discusses  the  rationale  for  considering 
each  problem  as  a test  item,  or  each  problem  as  a test.  The  latter 
offers  many  important  advantages  but  adequate  means  for  scoring  the 
records  are  not  currently  available.  It  is  expected  that  the  present 
investigation  of  sooring  will  be  of  value  in  the  treatment  of  each 
problem  as  a test.  ^ 
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improved  by  taking  the  response  record*  into  account  and  giving  par- 
tial credit  for  incomplete  eolutione  which  are  ‘'on  the  right  track.* 
In  an  electronic  c ircu.it  tuoh  partial  credit  can  be  earned  by 
leolatlng  > the  trouble  to  the  correct  stags.  Technicians  can  be 
given  etill  additional  credit  if  they  remain  in  the  correct  area 
after  they  have  Isolated  the  trouble  to  the  area. 

Response  records  also  provide  scores  on  negative  aspects  of  the 

19 

performance,  for  example.  Glaser  has  suggested  that  redundant 
checks,  or  checks  that  produce  no  new  information,  be  negatively 
weighted.  Isolation  ftp  an  incorrect  stage  or  functional  unit  and 
replacement  of  components,  that  are  not  faulty  also  seem  to  indicate 
wrong  or  inefficient  interpretations  by  the  technician  subject. 

There  arc  error  ohecke  which  could  not  conceivably  give  information 
about  any  trouble  (voltage  measurements  at  ground)  or  which  repre- 
sent faulty  or  inappropriate  ubo  of  test  equipment  (injection  of 
signals  into  power  supply). 

Hot  all  potential  scoring  parameters  may  bs  definitely  assigned 


'One  possible  criterion  for  isolation  consist*  of  a minimum 
ooneeeutive  number  of  itepe  in  the  eteg*  in  question. 

18 

This  same  form  of  analysis  may  be  carried  out  using  units 
•mailer  than  a stage.  Present  plane  include  an  analysis  of  the  re- 
sponse records  in  terms  of  "functional  units.*  functional  units 
are  designated  on  the  bates  of  a somewhat  arbitrary  breakdown  of 
eaoh  stage  Into  closely  related  elements,  for  example,  the  plate 
circuit  of  the  If  amplifier  is  a functional  unit.  The  radio  re- 
ceiver hat  eight  stages  and  forty-six  functional  units,  while  the 
radar  circuit  consists  of  ten  stages  and  forty-two  functional  unite. 

19  ~ 

'Glaser,  Robert  B,  A Proficiency  Test  Battery  for  folded 

Missile  Technicians.  Pittsburgh  JUusrican  Institute"  for  Research. 
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positive  or  negative  weights.  Instead,  many  measures  most  properly 
should  be  considered  as  scoring  hypotheses,  In  some  oases,  predic- 
tions of  the  positive  or  negative  nature  of  the  scores  can  he  made 

20 

on  the  basis  of  previous  research,  while  other  scores  are  frankly 
exploratory.  As  indicated  above,  the  records  are  being  scored  along 
these  parameters  in  order  to  empirically  determine  the  relationships 

among  these  and  more  conventional  measures.  One  such  parameter  is 

'''  \) 

t lae-to-solut ion.  It  may  be  argued  that  the  best  technicians  (by  def- 
inition) are  the  ones  who  accomplish  their  repairs  most  speedily.  On 
the  other  hand,  many  experienoed  technicians  urgs  caution  and  delibera- 
tion in  the  approach  to  the  problem.  Comparison  of  time-to-solution 
scores  will  reveal  whether  or  not  the  speedy  repairmen  are  generally 
more  successful  than  slow  technic Ians  under  the  circumstances  of  the 
AUfQNiST8  test. 

The  number  of  eteps  or  actions  accomplished  by  the  technician  dur- 
ing a problem  is  amenable  to  the  same  kind  of  analysis.  It  has  been 

proposed  that  a relatively  small  number  of  steps  is  Indicative  of 

21 

economy  or  efficiency.  However,  a small  number  of  eteps  may  not  be 
prediotive  of  other  measuree  of  performance.  These  relationships  must 
bs  empirically  investigated  before  it  is  assumed  that  economy  is  co- 
ordinate with  euooess. 

Various  measures  of  the  error  distance  or  degree  of  removal  of 

Saupe,  Joseph  L.  Trouble  8hoot ing  Hectronic  Equipment.  Urbanat 
Bureau  of  lducational  Research,  ’University  of  Illinois,  1954,  127  p. 
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the  responses  from  the  actual  site  of  the  trouble  may  be  useful* 

In  a mechanioal  trouble  shooting  situation*  Pattu,  Mech*  and  Kapos 
used  the  number  of  funotlonal  units  separating  the  response  from  the 
trouble  as  a basis  for  error  distanoe  scores.  This  same  system  can 
be  applied  to  electronic  equipment.  A similar  error  distance  score 
may  be  derived  in  a different  manner.  The  readings  for  a given 
problem  can  be  classified  as  critical  Information  readings  (lead 
directly  to  trouble)*  non-critlcal  information  readings  (provide 
useful  information  but  do  not  furnish  positive  localisation)*  and 
no  information  readings.  Each  response  can  then  be  weighted  accord- 
ing to  membership  in  one  of  the  three  classes  of  readings  and  all 
steps  averaged  into  a weighted  distance  Index* 

The  critical  information  readings  are  direct  clues  to  the 
faulty  component.  A score  of  the  critical  clue  utilisation  can  be 
determined  by  counting  the  number  of  times  that  critical  clues  were 
obtained  but  not  utilised,  (i*e. * without  the  indicated  subsequent 
responses  being  made). 

Since  all  the  checks  made  are  identifiable  from  the  response 
record*  it  is  possible  to  score  the  technician  in  terms  of  hypoth- 
eses tested  (parts  replaced)  as  a result  of  guesses.  If  a compon- 
ent is  replaced  or  adjusted  without  previous  checks  being  made  which 

if 

would  supply  clues  relative  to  that  part,  then  the  replacement  is 
counted  as  a guess. 


Pattu,  N.  A.,  Mech,  X*,  and  Kapos,  X,  Some  statistical  re- 
lationships between  selected  response  dimensions  and  problem-solving 
proficiency.  Psychol.  Monogr. , 1954,  6S,  Ho.  6 (Whole  Ho.  377). 


k tabulation  of  tha  frequency  of  types  of  checke  or  readings  can 
reveal  individual  differences  in  test  equipment  usage. 

It  may  be  useful  to  fratctioM^e  the  response  records  on  the  basis 
of  the  number  of  responses.  Tor  example,  splitting  the  performance  of 
each  man  into  quarters  will  allow  frequency  comparisons  of  entries 
in  different  functional  units  for  each  quarter  of  the  performance. 

Most  of  the  scoring  methods  mentioned  above  have  not  involved 
the  sequential  order  of  all  of  the  steps  taken  in  the  trouble  shooting 
task.  The  extremely  large  number  of  possible  orders  imposes  great  prac- 
tical difficulties  on  such  analyses,  and  yet  the  order  of  the  responses 
is  clearly  one  of  the  most  Important  features  of  the  performance.  Eval- 
uation of  the  technician's  method  or  solution  scheme  depends  on  the 
sequence  of  the  steps. 

Present  attempts  to  deal  with  the  sequence  of  responses  have  been 

along  two  lines.  The  first  approach  involves  the  behavioral  def ini- 
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tion  of  some  particular  trouble  shooting  system.  The  system  requires 
def inite  classes  of  responses  to  be  made  in  a certain  order.  Response 
records  are  then  examined  to  determine  the  degree  of  adherence  to  the 
system.  The  result  is  a score  whioh  expresses  how  nearly  the  techni- 
cian conforms  to  the  system  used  as  a standard.  If  this  kind  of  anal- 
ysis proves  feasible,  it  can  answer  such  questions  av  "Do  all  techni- 
cians use  the  same  trouble  shooting  method  for  a given  problem  or  typo 
of  symptom, * "How  many  trouble  shooting  methods  are  discernible  from 
AtJTOMASTS  data,"  or  "Bo  successful  technicians  employ  a method  which 

— — gy 

"Ideal"  systems  have  been  proposed  on  logical  grounds,  and  or- 
dered checking  sequences  are  often  suggested  by  equipment  manufacturers. 
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1b  different  from  that  of  unsuccessful  technic Ians* t If  the  latter 
question  can  he  answered  in  the  affirmative*  then  technicians  could 
he  graded  on  their  degree  of  adherence  to  the  successful  method* 

A second  approach  to  evaluating  sequence  consists  of  clas- 
sifying the  total  response  pattern  into  several  categories  of  types 

of  attack.  Sets  of  categories  have  been  applied  hy  Cornell,  Damrin, 
24  25 

and  Saupe,  and  hy  Saupe,  This  method  requires  tho  judgments  of 
several  experts  in  order  to  estimate  the  reliability  of  the  classi- 
fication* 

About  thirty  different  scores  are  now  being  applied  to  AUTO- 
liASTS  response  records.  Detailed  descriptions  of  the  various  scores 
and  the  relationships  between  them  will  be  presented  in  later  re- 
ports of  this  series* 


VI.  DISCUSSION 


The  results  of  the  administration  of  the  AUT01IA.STS  will  be  pre- 
sented in  detail  in  Technical  Report  No,  12.  The  ultimate  worth  of 
the  AUTOHASTS  must  be  judged  in  terms  of  its  results  and  their  con- 
tribution to  better  understanding  of  the  trouble  shooting  process 
and  its  measurement.  However*  at  this  point  it  is  appropriate  to 


;( 

Cornell*  Francis  G. , Damrin,  Dora  E, , and  Saupe,  Joe  L* 
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reexamine  the  objective*  sat  for  the  teat  format  (i.  a. , the  tea  ting  ma- 
chines and  procedures)  and  to  appraiae  the  extant  to  which  these  objec- 
tives have  been  achieved.  The  specific  teat  content  (i.e. , the  partic- 
ular circuits,  troubles,  scoring  systems,  and  psychometric  properties 
of  the  present  test)  will  be  evaluated  in  the  later  reports. 

At  the  outset  it  is  important  to  point  out  that  the  present  ma- 
chines must  be  regarded  as  prototype  models.  Competent  engineers  will 
be  able  to  devise  far  better  pieces  of  hardware.  Sven  the  present  in- 
vestigators see  many  opportunities  for  improvement  of  the  existing 
models.  Despite  this  faot,  the  machines  appear  to  have  done  the  job 
for  which  they  were  designed.  They  produce  permanent,  sequential  rec- 
ords of  the  steps  technicians  take  in  diagnosing  and  correcting  diffi- 
culties in  electronic  equipment.  They  present  the  technician  with  an 
"open-ended”  and  realistic  problem,  i.e. , they  make  a large  number  of 
items  of  information  available  to  him  but  require  the  technician  to 

supply  his  own  procedures  for  sampling  and  interpreting  the  information. 

}/  '• 

They  do  not  restrict  the  behavior  of  the  men  in  such  a way  that  one 
particular  method  of  trouble  shooting  is  favored  over  another.  The 
men  are  free  to  trouble  shoot  in  the  manner  to  which  they  are  ac- 
customed* 


The  machines  are  portable  and  rugged.  They  make  possible  group 
administration  of  this  type  of  trouble  shooting  problem.  They  also  ex- 
pedite the  trouble  shooting  procedures  so  that  the  same  problems  which 
required  a 35-minute  cutoff  time  in  the  job  sample  format  may  be  suc- 
cessfully administered  with  a 15  or  10  minute  cutoff  time  in  the  AUTO- 
MASTS form.  The  test  may  be  administered  by  persons  without  special 
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training  in  electronics* 

The  machines  may  be  used  tor  a wide  variety  of  problems  in- 
volving different  circuits.  In  order  to  employ  new  circuits  or  new 
problems,  only  the  reference  materials  and  the  paper  portion  of  the 

K 

problem-information  dish  need  be  changed.  For  this  reason  the  ma- 
chines will  be  adapted  for  further  research  as  indicated  belov/. 

Men  who  have  taken  AUTCHASTS  problems  report  that  they  "feel" 
they  are  trouble  shooting.  Several  have  suggested  applications  for 
the  test.  Many  pointed  out  possibilities  for  the  test  as  a train- 
ing device.  Most  of  the  men  indicated  that  thiB  form  of  test  covered 
a part  of  their  jobs  that  was  not  covered  by  an  extensive  battery  of 
job-oriented  pa per- and- pane il  tests* 

Despite  the  fact  that  the  AUT01IASTS  seems  to  have  lived  up 
to  expectations  one  should  not  lose  sight  of  thefhet  that  it  is 
still  relatively  untested  and  that  an  important  part  of  the  research 
yet  to  be  done  with  respect  to  the  AUT0MAS7S  is  to  determine  its 
limitations.  In  addition  to  this  general  methodological  research, 
future  plans  for  the  use  of  the  AUTOliaSTS  technique  include  the 
follow ingt 

1.  Develojme^  of  ££££>  adequate  testing  machines.  In  gen- 
eral this  development  will  be  carried  out  by  groups  especially 
oriented  toward  hardware  development  although  modifications  of  the 
five  pilot  models  will  be  made  as  necessary  to  conduct  elements  of 
the  research  mentioned  below.  It  is  felt  that  a completely  self- 
administering  form  of  the  test  (e,g. , with  automatic  problem- 
changer)  can  be  and  should  be  developed.  If  this  transpires,  and 
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If  a sufficient  number  of  these  machines  are  produced,  vast  quantities 
of  data  will  become  available  for  study,  this  will  be  a very  important 
contribution  to  the  analysis  of  trouble  shooting  behavior,  the  paucity 
of  such  information  is  one  of  the  reasons  that  many  of  the  problems  in- 
volved in  the  analysis  and  measurement  of  electronics  trouble  shooting 
cannot  be  attacked  at  this  time,  When  a large  number  of  response  rec- 
ords are  available  numerous  hypotheses  regarding  the  nature  of  trouble 
shooting  behavior  may  be  subjected  to  test. 

2.  Development  o£  a set  of  problems  based  on  Kayy  equipment.  A 
second  projected  use  of  the  AUTOtASTS  testing  system  is  to  develop  a 
eet  of  problems  based  on  operational  Navy  equipment  for  use  in  the  ma- 
chines* Additional  plans  are  being  made  for  the  development  of  prob- 
lems involving  so-called  **  fundamental**  circuitB.  When  problems  have 
been  developed  on  a number  of  different  kinds  of  circuits  and  equip- 
ments it  will  be  possible  to  investigate  such  important  areas  as  the 
transfer  which  occurs  between  trouble  shooting  one  type  of  electronic 
equipment  and  another* 

3.  Evaluation  of  the  AUPOIASTS  as  a training  device.  Plans  are  - 
currently  being  formulated  for  the  conduct  of  training  research  with 
the  present  AUTOMASTS,  The  objectives  of  this  research  would  be  tvo- 
foldl  first,  to  determine?  the  usefulness  of  the  AUTOIASTS  Test  as  a 
training  device,  and  second,  if  the  device  proves  to  be  useful  as  a 
trainer,  to  develop  a training  program  which  can  be  used  aboard  ship 
and  in  training  schools  to  supplement  for  trouble  shooting  the  actual 
equipments. 


4,  Determination  of  the  relat  Unship  between  duty  assignment 
and  trouble  shooting  skill,  Present  plans  also  include  the  possi- 
bility of  employing  the  AUTOIIASTS  to  determine  the  extent  to  which 
men  have  lost  the  ability  to  trouble  shoot  as  a result  of  periods 
spent  away  from  trouble  shooting  assignments.  A similar  problem 
to  be  investigated  is  the  effects  of  protracted  specialised  assign- 
ments (such  as  a tour  of  duty  at  a land-based  transmitter  station) 
upon  trouble  shooting  performance, 

5*  Comparison  of  trouble  shooting  methods.  The  response  rec- 
ords produced  by  the  AUTOIIASTS  provide  a suitable  basis  for  compar ac- 
tive evaluation  of  various  trouble  shooting  methods  or  systems, 
Matched  groups  of  experimental  subjects  may  be  trained  to  trouble 
shoot  according  to  specified  procedures.  At  the  conclusion  of  the 
training  period  the  entire  group  can  be  tested  on  the  AUTOIIASTS  in 
order  to  evaluate  the  efficacy  of  the  different  procedures.  Under 
appropriate  conditions  of  experimental  control,,  this  procedure  will 
also  uncover  interactions  between  trouble  shooting  methods  and 
problem  types  (if  such  interactions  exist), 

6.  Analysis  of  trouble  shooting  problems.  Since  the  AUIO- 
HASTS  permits  experimental  control  of  the  problem  information  fur- 
nished the  technician  it  is  possible  to  systematically  investigate 
such  factors  as  v/hat  makes  a problem  hard  or  easy;  or  what  role 
extensiveness  of  cues,  circuit  complexity,  and  type  of  circuit  play 
in  tho  trouble  shooting  process, 

*****+********* 
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APPENDIX 


Ih is  appendix  contains  schematic  diagrams  of  the 
equipments  currently  beins  used  for  i.TBCHASTS  trouble 
shooting  problems,  The  diagrams  shovu  here  haye  been 
drastically  reduced  in  size  from  the  schematics  actu- 
ally used  in  AliTOHACTS  tenting,  On  both  schematics, 
test  point  numbers  are  in  boxes  and  component  numbers 
are  circled* 

These  schematics  are  specially  adapted  versions  of 
the  original  schematics  published  by  the  Philco  Corpora- 
tion in  HAVSHIPS  900170  Electronics  training  Series* 
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BASIC  RADIO  RECEIVER  CIRCUIT  DIAGRAM 


GENERATOR  CIRCUIT  DIAGRAM 


